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The EVALUATION CENTER, an agency of the College of 
Education, is committed to advancing the science and 
practice of educational evaluation. More specifically, the 
purpose of the Center is to increase education’s capability 
to obtain and use information for planning, programming, 
implementing and evolving educational activities. To serve 
this purpose, the Center’s interdisciplinary team engages 
in research, development, instruction, leadership and 
service activities. 

HISTORY ^ ^ 

The origin of the present Center traces back to 
the establishment of the Ohio State University Test Devel- 
opment Center in 1962. Due to the urgent need for a more 
comprehensive approach to evaluation than that afforded 
by standardized testing, the Test Development Center \was 
expanded in 1965 into the present Evaluation Center \which 
is concerned \with many modes of evaluation in addition to 
standardized testing. Ho\wever, test development remains 
an important part of the Evaluation Center program. 
GOALS 

The broad objectives of the currently constituted 
Csntsr 0 rs* 

to increase scientific kno\wledge of educational evaluation 
and planning; 

to develop evaluation strategies and designs; 
to develop evaluation methods and materials; 
to provide instruction in evaluation; 

to disseminate information related to educational evalua- 
tion; 

to assist educationists in evaluating their programs. 
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ORGANIZATION 

To serve its complex objectives, the Center has 
developed an interdisciplinary team. Currently, the staff 
of the Center consists of fifty-four members, including 
five professorial positions, plus a varying number of visiting 
faculty. The staff and visiting professors bring expertise 
from the fields of economics, education (administration, 
curriculum and supervision, elementary and secondary 
school teaching, evaluation, mathematics, planning, re- 
search methodology, and tests and measurement), psy- 
chology, sociology, systems analysis, and urban planning. 
The Center is organized into four divisions: Administration 
and Program Development; Leadership in Evaluation; Re- 
search in Evaluation; and Test Development. The Center 
is administered by a director and an associate director 
foi each division. 

STAFF 

Daniel L. Stufflebeam, Director 

Michael S. Cald\well, Associate Director 

Administration and Program Development 

Ed\win G. Novak, Associate Director 
Research in Evaluation 

Jack M. Ott, Associate Director 
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INTRODUCTION 

This paper represents a first approximation attempt at devising 
a "way of looking" at educational planning and the assessment of this 
planning. An adequate coverage, of the points mentioned here would 
require a document the scope and depth of which this writer Is not 
capable of producing at present. However, the purposes of this paper 
are heuristic in nature and are not so ambitious as to suggest that 
I have answers to the problems of educational planning and plans 

assessment* 

These remarks are not intended as an apology or disclaimer 
are to request reactions to what this paper says-or does not say. 



Michael S. Caldwell 



Input Evaluation and Educational Planning 



American education at all levels has become enamored with the con- 
cept of change. Both the literature and the dialogue of the field 
are liberally spiced with such terms as change, innovation, planned 
change, and planning for change. A cursory examination of the liter- 
ature and an analysis of the concept of change as revealed by this 
literature highlights one crucial factor-the key aspect of the con- 
cept is not change but planning. While most educators are quite 
willing to accept the inevitability of change, they are not willing 
to accept "change for the sake of change." Thus, the focus, and in 
this writer*s opinion, a proper one, is upon giving proper direction 
and substance to change through sound educational planning. 

The Ohio State University Evaluation Center is concerned with 
efforts to improve American education and assumes that effective planning 
is crucial In any attempt to effect such improvement. The CIPP Evalu- 
ation Strategy! which is the basis for much of the Center's activities 
is composed of four stages— one of which relates directly to educa- 
tional planning. Input Evaluation refers to the assessment of the 
inputs which one is able or willing to invest in order to realize certain 
outputs, i.e., the desired change or improvement. In an educational 
setting inputs are normally programs or projects both of which are, 
or should be, based on sound planning. Thus, Input Evaluation may 
be thought of as a ki nd of plans, assessment. 



^Daniel L. Stuff lebeam. "The Use and Abuse of Evaluation in Title 
III," Theory Into Practice, VI (June, 1967)» PP* 129-131* 
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The purpose of this paper Is two-fold-(l) to provide a framework 
within which the concept of Input Evaluation might be viewed and (2) 
to suggest an approach for actually implementing an Input Evaluation. 
In the broadest sense there are only four major steps in instituting 
any kind of change or improvement in an educational setting, i.e., 
determine what should be done, determine how to do it, do it, and 
determine the impact of what was done. These steps suggest a system 
such as the one depicted in Figure I below. 



Figure I Educational Improvement Cycle 
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Obviously, the schema depicted in Figure I is greatly over simpli- 
fied and is, in fact, so broad that it is meaningless in an operational 
sense. Furthermore it completely disregards a large body of inform- 
ation relative to the concept of change. However, the schema does 
fulfill one important function. It provides a quick overview of the 
total process and provides handles which allow one to locate himself 
in the total process. 

The terms strategy and plan are used throughout the discussion 
which follows and require some explanation. A strategy is a general 
approach to a situation and a plan is a highly cxpl id t—* possibly even 
programmed*— set of activities through which the strategy is operation- 
alized. If a superintendent of schools wishes to institute some change 
in response to a crucial need— *say, low achievement of disadvantaged 
children in reading— he may employ one (or more) of several strategies, 
e.g. , upgrade his present teaching staff, devote a larger portion of the 
school day to reading, hold weekend or evening reading enrichment 
classes, employ a remedial reading teacher, etc. Suppose the superin- 
tendent chooses to upgrade his present teaching staff as a general 
strategy. He then must develop several alternative preliminary plans, 
e.g., operate an internal in-service education program, request a 
nearby college to provide an in-service program, have his teachers 
enroll in formal college courses, etc. If he chooses (from among the 
alternative preliminary plans) to operate an internal in-service pro- 
gram, he then must develop a very explicit action plan which will 
detail exactly how such a program is to be operated. 
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A College Dean who Is attempting to promote research and develop- 
ment activities is faced with a similar situation. He must select 
from several general strategies, e.g., employ new research and develop- 
ment oriented staff members, create a bureau of research, charge some 
college officer to promote research and development activities and give 
technical support to staff members, etc,^ Regardless of which of 
these general strategies he chooses, and he may choose some combination 
of strategies, he must develop specific procedures, I.e,, action plans, 
through which the goals of the strategy can be realized. 

State departments of education are also faced with planning situ- 
ations which call for the selection of general strategies and specific 
action plans. For example, a state department of education might 
perceive a need to improve urban education programs for disadvantaged 
children. Several strategies are available on the state level, e.g., 
adijjstment of certification requirements for teachers of the disadvan- 
taged, adjustment of state aid formulas to provide additional state 
funds to urban areas, increase the percentage of total state Elementary 
and Secondary Act Title I funds for urban areas, etc. After deciding 
upon a general strategy, detailed plans would have to be developed to 

implement the strategy. 

Once an educational system or agency has focused on a need which 
requires attention, what steps must be taken to Insure sound planning 

Vor a more detailed discussion of some of the strategies which 
are employed to promote research and development activities in univer- 
sity settings see Seiber, Sam D., The Organization of Educational Re- 
search > Bureau of Applied Social Research, Columbia University, I966. 
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for attacking that need? Figure II is a schema of a process which 
might be used in the second stage of the Educational Improvement Cycle, 
i,e., developing strategies and plans for meeting selected needs. This 
schema is intended only as a guide and will require a great deal of 
explication In the context of the specific agency or system before it 
can be implemented, e.g. What person or unit is responsible for the 
various steps?; Who makes decisions at various stages?; What is the 

schedule? 

The program selection process begins (step I) when a need is 
identified and a decision is made to explore the possibilities of 
meeting that need. For example, suppose the board of education of a 
metropolitan school system decides to attack the problem of ^ facto 
segregation. The board would then charge the superintendent of 
schools to develop a program proposal memo (step II). A program pro- 
posal memo is a brief document which might contain (a) a description 
of the need and a rationale for attacking the need, (b) the broad 
goals to be achieved and (c) alternative strategies (with strengths 
and weaknesses of each) for achieving the broad goals. 

After the superintendent has developed this memo proposing alter- 
native strategies, e.g., revise existing attendance area boundaries, 
bus children across attendance area lines, or Institute open enrollment 
in the district, the memo and the strategies are reviewed (step III). 
Since the various review steps (III, VI and IX) are treated in some 
detail later, no explanation of these steps will be offered at this 
point. At step IV the primary decision-maker. In this case the board of 
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education, has three alternatives, (1) choose from the alternative 
strategies, (2) in the event that none of the alternatives is satis- 
factory, ask the superintendent to suggest additional strategies, or 
(3) terminate the process. 

If the board of education selects a general solution strategy- 
say, busing children across attendance area boundaries— prel imi nary 
plans for achieving the goals of the strategy must be developed (step 
V), The preliminary plans document is much more explicit than the 
program proposal memo and should contain (a) a description of the need 
and a rationale for attacking the need, (b) the broad goals to be 
achieved, (c) a description of and justification for the selected strat- 
egy, and (d) alternative preliminary plans to implement the strategy. 
These preliminary plans should include, at least in first approximation 
form, information relative to (1) the person or unit responsible for 
implementing the plan, (2) the resources which will be necessary, (3) 
a schedule of activities, (4) cost estimates and (5) estimated out- 
comes • 

The review of the preliminary action plans document (step VI) is 
much like the review in step III except that as plans become more ex- 
plicit review techniques become more refined and rigorous. At step 
VII the decision-maker has three alternatives, i.e. , choose from the 
alternatives, recycle and ask for additional alternatives, or decide 
to terminate the process. 
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After a decision is reached regarding which preliminary plan or 
combination of preliminary plans will be used, e.g,, which children 
will be involved, how children will be selected, which school buildings 
will be involved, etc,, a fully explicated action plan must be developed 
(step VIII), An action plan which is ready for implementation should 
contain (a) a description of the need which the action plan is designed 
to meet, (b) the goals (expected outcomes) of the plan, (c) the rela- 
tionship of the expected outcomes of the plan to the need, i.e,, to 
what degree will the need be met if the goals of the action plan are 
achieved, (d) the functions and/or activities which the plan requires, 
(e) the resource and concomitant cost requirements of the functions 
and activities, and (f) the scheduling of activities. Perhaps the 
best definition of an action plan which is ready for implementation 
•s a pi an which i s detai led and programmed to such an extent that 
some i ndi vidual who has not been i nvo 1 ved i n the planning will be able 
to implement the plan and wi 1 1 most 1 i kel v be able to achieve the 
goal s of the plan . 

These alternative action plans will have a final review (step IX) 
the purpose of which will be to search for programming inconsistencies 
and to assess and compare the plans in terms of cost/effectiveness con- 
siderations, At step X there will be only two decision alternatives 
in most cases— select from alternative action plans or recycle for 
minor revisions and plan refinement. While a decision to terminate is 
always possible, it is not likely at this point unless some changes 
and/or pressures which are external to the process are brought to bear. 
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Steps XI, XII, XIII and XIV are self-explanatory. However, funding 
considerations are crucial and must be regarded as possible constraints 
throughout the process. In other words, unless some ”feel^' for the 
general funding situation is fed into the process (note the dotted lines 
in the schema) from the very begi nning^ the whole activity may be just 
an academic exercise— and a costly one at that. 

Figure II, The Program Selection Process, portrays the manner in 
which an agency might develop and assess strategies and plans for 
attacking an identified need. Steps IV, VII and X are key decision- 
making points in the process and sound decision-making at these points 
must be based on valid and reliable information. Such information 
must be gathered, analyzed and reported at the review points (steps 
III, VI and IX) which immediately precede the steps at which decisions 
are made. 

The purpose of the review steps is to provide information for 
decision-making relative to approaches to meeting needs. Obviously, 
this is just another way of saying that Input Evaluation provides in- 
formation for answering the questions which decision-makers might 
(or should) ask. How does one generate crucial questions about strat- 
egies and plans for meeting needs? If one posed all of the possible 
questions relative to the viability of a strategy or action plan for 
meeting a given need, these questions could all be grouped under one 
or the other (or possible a combination of the two) of two general 
criteria— desirabi 1 ity considerations and feasibility considerations. 
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While criteria classes are useful in the generation of key 
questions, desirability and feasibility are too broad to be of much 
value. However, by applying desirability and feasibility considerations 
to strategies in terms of what the strategy is for, where it will be 
implemented, by whom it will be implemented and in comparison with other 
alternative strategies, it might be possible to generate more specific 
and more useful criteria. The idea is to juxtapose the strategy or 
plan with the identified need, with the context, with the implementing 
agency and with alternative strategies or plans and to consider the 
general criteria of desirability and feasibility in terms of these 
factors. This will serve to highlight specific criteria which might 
be applied to the proposed strategies and plans. There are at least 
eight (and probably more) criteria which might be generated in this 
manner, (1) relevance, (2) legality, (3) congruence, {k) legitimacy, 

(5) compatibility, (6) balance, (7) practicabi 1 ity and (8) cost/ 
effectiveness. These criteria are not totally discrete and there is 
some overlap in certain areas. They are, however, most useful in 
terms of one of their primary purposes, i.e., to assist in the 
generation of crucial questions which the decision-maker should con- 
sider. 

Figure 111 is a suggested framework for conducting the plans 
assessments ^stages Ml, VI and IX) in the Program Selection Process, 
The illustrative material contained in the cells of the framework 
refers only to strategies and, therefore, appears to refer only to 
the plans assessment (stage III) for evaluating strategies. However, 
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Figure III 

An Approach to Plans Assessment 
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by substituting the term prellmilnary plan for the term strategy, the 
framework has application at stage VI and by substituting the term 
action plan. It has application at stage IX, 

While It Is true that certain criteria seem to be more applicable 
at certain stages of the Program Selection Process than at others, ex- 
treme caution should be exercised In completely disregarding a criterion 
at any level. For example, If one applies relevance at stage III and 
does not consider practicability he may expend a great deal of effort 
developing a strategy which cannot be Implemented, In other words. 

In order to avoid costly unproductive mental exercises all criteria 
should be checked at all review steps. If a criterion Is considered 
and a decision Is made that the criterion does not apply at that point, 
no harm is done but to overlook a criterion which does apply could 
prove disastrous. 

The primary difference between the application of the criteria 
at various plans assessment stages Is one of degree. Strategies 
(evaluated at stage III) are more global than are preliminary plans 
(evaluated at stage VI) which, in turn, are not as explicit as highly 
detailed action plans (evaluated at stage IX), As plans become more 
explicit and detailed, questions and approaches to answer these 
questions become much more sophisticated. 

Relevance This criterion refers to the degree to which the need 
will be met If the purposes (end-products) of the strategy or action 
plan are achieved. The criterion of relevance serves to keep planners 
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*'on-track“ in developing strategies and plans. In order to avoid 
statements such as "I achieved all of the objectives of my plan but 
didn't meet the need," planners must apply this criterion as rigor- 
ously as possible at all stages of the Program Selection Process. 

Legal i tv This criterion grows out of the juxtaposition of the 
strategy or plan and the context in which it is to be Implemented. 

On the face of it, this seems to be a simple criterion to apply; 
however, there are strategies and plans which may be legal in one 
context but not legal in another. For example, a strategy or plan 
which included a biology curriculum program built around certain 
concepts of human solution might be illegal in states (there are 
presently two such states) which have laws regarding the teaching of 
evolution. Also, difficulties have been experienced in the imple- 
mentation of certain Elementary and Secondary Education Act programs 
because of laws and the interpretation of these laws In some states 
regarding the separation of church and state. 

Congruence The criterion of congruence relates to the degree 
to which a proposed strategy or plan is consistent with the value 
system(s) of the context in which it is to be implemented. For 
example, it has been proposed that one possible solution strategy for 
food shortages In India would be to slaughter sacred cows. It is 
contended that this strategy would have a positive impact on the need 
in at least two ways**"(l) the animals themselves could be used for 
food and (2) the crops which these animals eat or destroy as they 
roam the countryside could be used for human consumption. While such 
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a strategy might be considered highly desirable In terms of Its 
relevance to the need, it is clearly in opposition to certain values 
which are strongly held by significant proportions of the indian 

population. 

It is quite possible that strategies and plans which include 
highly innovative curriculum practices and/or teaching techniques 
might enjoy success in one school district and experience failure in 
another. One possible hypothesis for the occurrence of such a situ- 
ation would be that such practices and techniques are consistent with 
the value system(s) of one context and are less consistent or even 
inconsistent with the values of the other. The point of these 
illustrations is simple, a strategy or plan must be considered in 
terms of its “fit" with the context in which it is to be implemented. 
Legitimacy This criterion grows out of Juxtaposing the strategy 
with the unit (individual or agency) charged with implementing that 
strategy and refers to whether the strategy is within the purview 
of the implementing unit.* The application of this criterion points 
up some interesting and highly crucial types of questions regarding 
levels and types of responsibility and authority , i.e., do I 






In some situations certain of the criteria in Figure III should 
be applied by two different agencies in two different frames ot re- 
ference, e.g. , by the agency, such as a public school system, within 
which the plan will be implemented and by the agency, such as a 
college or university unit, which will implement the plan. 
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(individual or agency) have the necessary authorization to implement 
this plan? For example, in the previously mentioned ^ fa^Jto 
segregation situation, while the alteration of attendance area 
boundaries might be considered as appropriately within the realm of 
the board of education's authority, it would certainly not be within 
the purview of a building principal's authority. 

The application of the criterion of legitimacy becomes much 
more complex when more than one agency is involved in implementing the 
plan. There are numerous examples which ope can cite in which a 
public school system and some unit of a college of education collab- 
orate on a plan which is implemented within the public school system 
by both agencies, 

C ompatibil Itv The criterion refers to the compatibility of the 
proposed strategy In terms of the purposes of the implementing agency. 
While the criterion of legitimacy relates to the question, "Can 1 do 
this?", compatibility is directed toward the question, "Should I do 
it?". For example, public school systems often ask for assistance 
from some unit in a university (most usually a college or school of 
education). The university unit must determine if providing such 
services would be appropriate in terms of its own goals, purposes and 

program foci. 

Balance The criterion of balance assumes that the settings In 
which solution strategies will be applied are multi -objective and 
are composed of numerous program elements (sub-systems). For example. 
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the total program of an educational agency is composed of discrete 
but interrelated substantive and skill elements such as reading, math 
and social studies. One possible solution strategy for a problem in 
social studies might be to invest all available resources into up- 
grading the program in this area. Obviously such a strategy would, 
while it might solve the social studies problem, seriously imbalance 
the total school program. 

The above illustration is highly unlikely to occur when con- 
sidered only in terms of a specific decision at a specific point in 
time. It is however a realistic example if decisions made today 
have implications for decisions made at some future point. While 
the classic illustration of this point involves future cost impli- 
cations of decisions made in the 1950*s to develop certain types of 
military aircraft, the Elementary and Secondary Education Act of 
1965 provides a concrete illustration in the field of Education. 

The guidelines for Title III projects include a commitment require- 
ment from local educational agencies involving "phasing out Federal 
support over a three-year period through gradual cost absorption by 
local or other funding,"^ Thus, local educational decision-makers, 
when they accept federal funding for Title 111 programs, are 
commiting future local resources to continue programs which are 
initiated at the present time. It is conceivable that such commit- 
ments, if met, could result In local resource allocation concentrations 




1 Title 111 guidelines p.9. 
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which might create serious system balance problems in three to five 
years. It is therefore imperative that educational planners when 
applying the criterion of balance not only consider immediate 
implications but also engage in predictive studies which would give 
indications of future implications of present decisions. 

Pract i cabi 1 i tv The criterion of practicability refers to how 
realistic the proposed solution strategy is in terms of achieving 
its stated purposes. In other words, what conditions must exist, to 
what degree must they exist and what are the chances that they will 
exist in order for this solution strategy to be successful? 8n order 
to apply this criterion and to assess the strategies and plans in 
terms of it, the education planner must be aware of six major types 
of constraints which might operate as barriers to success— (1) state 
of the art constraints, (2) resource availability constraints, (3) 
logistical constraints, (4) internal (organizational) constraints and 
(5) external (contextual) constraints. While it is true that these 
classes of constraints are not discrete and do overlap, it is useful 
to highlight them because of their crucial ity to the educational 
planner as he applies the practicability criterion. 

State of the Art Constraints It is conceivable that 
a solution strategy might require some piece of hard- 
ware, e.g,, an individual computer, and accompanying 
software, e.g., a learning program. Some other solu- 
tion strategy might require a standardized social studies 
instrument to be administered to children learning to 
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read by the I.T.A. Method* Do such items of hardware 
and/or software exist? Have techniques necessary to 
implement some crucial phase of the strategy been devel- 
oped? These questions are indicative of information 
which the educational planner needs in order to make 
sound judgments regarding potential state of the art 
constraints. 

2, Resource Availability Constraints The educational 
planner must have information regarding the resources, 
e.g., men and material, necessary to Implement th- 
proposed solution strategy and must assess the avail- 
ability of such resources. If such resources are 

not on hand, can they be procured and, if so, are 
funds available to do so? 

3, Logistical Constraints Broadly defined, logistical 
constraints are those associated with the trans- 
portation and scheduling of resources, i*e,, having 
the right resources. In the right amounts, at the 
right place and at the right time. In order for the 
planner to assess the programming of a solution 
strategy, he must have information which gives him a 
picture of the total activity. Various networking tech- 
niques are valuable tools for providing such information.^ 

^For a comprehensive examination of the uses of such techniques 
see H, S, Woodgate, Planning by Network , 1964. 
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4, Internal (organizational) Constraints A particular 

solution strategy may require close coordination and 
cooperation among several units and/or individuals ^ 

within the implementing agency. For example, an after- 
school study center might involve such cooperation 

and coordination among school building maintenance 
personnel, bus drivers, teachers, etc. If no commu- 
nication bridges have been built or, if for some 
reason the organizational structure of the school 
inhibits such communication, a number of problems 
might arise. 

5, External (contextual) Constraints In all contexts 
there are forces and/or specific organizations which 
might place constraints on a given solution strategy, 
e.g,, political forces, civil rights groups, religious 
groups, parents, etc. For example a solution strategy 
which involves evening study centers for young children 
might raise serious security questions from parent 
groups and from police authorities, 

Cost/Effect i veness This criterion grows out of juxtaposing 
the proposed strategy with alternative strategies. Translated 
freely the application of this criterion should provide decision- 
makers with the indications of which of several alternative 
strategies will give "more bang for the buck," Decision-makers 
need to compare alternative strategies in terms of what benefits 
(outputs) are expected and what these benefits will cost (inputs). 
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In determining the benefits to be derived from a given strategy, 
a distinction should be made between gross benefits and real benefits. 
Real benefits are determined by the following formula— gross Increments 
less trade-off decrements - real benefits. Gross increments refers to 
the total benefits to be realized from the implementation of the plan. 
Decrements are those things which must be surrendered in order to im- 
plement the plan. For example, a plan which requires more teacher 
time and student time to be devoted to reading may realize reading gains 
at the expense of gains In some other curriculum area, A second type 
of decrement is much more nebulous but is real just the same. Any 
decision to implement a strategy which requires resources (and all do) 

Is also a decision not to devote those same resources in some other 
part of the system, if it were possible to determine all possible com- 
binations of resource usage in all possible combinations of system 
elements, it might be possible to quantify this second type of decre- 
ment, Obviously, this situation will never occur and the decision- 
maker must live with the fact that any decision to commit resources is 
made without ful 1 knowledge of what is being given up. 





